Speeding up gradient-based algorithms for sequential games
نویسندگان
چکیده
First-order (i.e., gradient-based) methods for solving two-person zero-sum sequential games of imperfect information have recently become important tools in the construction of game theory-based agents. The computation time per iteration is typically dominated by matrix-vector product operations involving the payoff matrix A. In this paper, we describe two techniques for scaling up this operation. The first is a randomized sampling technique that approximates A with a sparser matrix Ã. Then an approximate equilibrium for the original game is found by finding an approximate equilibrium of the sampled game. The second technique involves the development of an algorithm and system for performing the matrix-vector product on a cache-coherent Non-Uniform Memory Access (ccNUMA) architecture. The two techniques can be applied together or separately, and they each lead to an algorithm that significantly outperforms the fastest prior gradient-based method.
منابع مشابه
Algorithms for abstracting and solving imperfect information games
Game theory is the mathematical study of rational behavior in strategic environments. In many settings, most notably two-person zero-sum games, game theory provides particularly strong and appealing solution concepts. Furthermore, these solutions are efficiently computable in the complexity-theory sense. However, in most interesting potential applications in artificial intelligence, the solutio...
متن کاملGradient-Based Algorithms for Finding Nash Equilibria in Extensive Form Games
We present a computational approach to the saddle-point formulation for the Nash equilibria of two-person, zero-sum sequential games of imperfect information. The algorithm is a first-order gradient method based on modern smoothing techniques for non-smooth convex optimization. The algorithm requires O(1/ ) iterations to compute an -equilibrium, and the work per iteration is extremely low. Thes...
متن کاملA Gradient-based Approach for Computing Nash Equilibria of Large Sequential Games
We propose a new gradient based scheme to approximate Nash equilibria of large sequential two-player, zero-sum games. The algorithm uses modern smoothing techniques for saddle-point problems tailored specifically for the polytopes used in the Nash equilibrium problem.
متن کاملSpeeding up learning automata based multi agent systems using the concepts of stigmergy and entropy
Learning automata (LA) were recently shown to be valuable tools for designing Multi-Agent Reinforcement Learning algorithms and are able to control the stochastic games. In this paper, the concepts of stigmergy and entropy are imported into learning automata based multi-agent systems with the purpose of providing a simple framework for interaction and coordination in multi-agent systems and spe...
متن کاملOptimal Linear Precoding Strategies for Wideband Non-Cooperative Systems based on Game Theory-Part II: Algorithms
In this two-part paper, we address the problem of finding the optimal precoding/multiplexing scheme for a set of non-cooperative links sharing the same physical resources, e.g., time and bandwidth. We consider two alternative optimization problems: P.1) the maximization of mutual information on each link, given constraints on the transmit power and spectral mask; and P.2) the maximization of th...
متن کامل